Presence and authentication for media measurement

ABSTRACT

A method, executed by a processor, is used to determine presence of a viewer at a media device. The method includes receiving viewer biometric data captured by a biometric capture device associated with the media device; determining a category of the viewer based on the captured viewer biometric data; comparing the captured viewer biometric data to a reference to determine a possible identity of the viewer, by: determining a presence probability for the viewer based on a match between the biometric data and the reference, and determining a confidence level for the probability; and when the probability and confidence level equal or exceed a threshold, determining the viewer is present at the media device.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of priority under 35 U.S.C. § 120 asa continuation of U.S. patent application Ser. No. 14/933,937, filedNov. 5, 2015, which claims the benefit of priority under 35 U.S.C. § 120as a continuation-in-part of U.S. application Ser. No. 13/843,559, filedMar. 15, 2013, each of which are hereby incorporated by reference intheir entirety.

BACKGROUND

Voice and gesture control systems, mechanisms, and devices are in use tocontrol access to various media devices. For example, television remotecontrol devices may incorporate a voice activated feature that allows aviewer to turn the television, and change channels and volume. The voiceactivation feature also may be used to sign on to a service, purchase apay-per-view movie, and complete other transactions that in the pastrequired manual entry using buttons on the remote control. Use of voicecontrol may enhance the viewer's television viewing experience by doingaway with cumbersome on screen interfaces and eliminating the need tofind a specific button in a sea of remote control buttons. In additionto voice recognition, some systems incorporate gesture control, wherebya small camera on a television, for example, captures viewer gestures toidentify an action requested by the viewer, such as to turn thetelevision volume down.

SUMMARY

A method, executed by a processor, is used to determine presence of aviewer at a media device. The method includes receiving viewer biometricdata captured by a biometric capture device associated with the mediadevice; determining a category of the viewer based on the capturedviewer biometric data; comparing the captured viewer biometric data to areference to determine a possible identity of the viewer, by:determining a presence probability for the viewer based on a matchbetween the biometric data and the reference, and determining aconfidence level for the probability; and when the probability andconfidence level equal or exceed a threshold, determining the viewer ispresent at the media device.

DESCRIPTION OF THE DRAWINGS

The detailed description refers to the following Figures in which likenumerals refer to like items, and in which:

FIG. 1A-1C illustrate example environments in which viewer presence andauthentication are enabled;

FIGS. 2A and 2B illustrates an example server-side measurement systemthat determines or uses presence and authentication information;

FIG. 3 illustrates and example client-side system that determines thepresence and authentication of a viewer; and

FIGS. 3-8 are flowcharts illustrating example viewer presence andauthentication processes as executed by the systems of FIGS. 2A and 2B.

DETAILED DESCRIPTION

Media consumption is becoming increasingly personalized. This trend israpidly moving from classic internet experience to all manner of devicesin the home, including televisions. However, many televisions (and gameconsoles) are “shared” media devices. Adding personalization has thepotential to greatly increase the value of a viewer's experience, but abarrier to making this seamless is the ability to authenticate theviewer. New television services, particularly Internet Protocoltelevision (IPTV)-related services may allow the viewer to sign in tothe service. Currently, such a sign-in may involve use of a cumbersomeuser interface or text entry system. Furthermore, there is no easy wayfor the viewer to log out for a short period.

A related problem exists in the media measurement space. For measurementpurposes, it is important to know which viewer is watching television atany given time (known as “presence”). This problem may be solved byasking viewers to log in and out using a special remote control.However, this solution may present a significant cost to implement andmay lead to compliance issues. Even if a truly “personalized” servicewere available, such a service might not be used with enough consistencyfor media metrics measurement.

Voice and gesture control systems, mechanisms, and devices are in use tocontrol access to various media devices. For example, television remotecontrol devices may incorporate a voice activated feature that allows aviewer to turn the television, and change channels and volume. The voiceactivation feature also may be used to sign on to a service (as notedabove), purchase a pay-per-view movie, and complete other transactionsthat in the past required manual entry using buttons on the remotecontrol. Use of voice control may enhance the viewer's televisionviewing experience by doing away with cumbersome on screen interfacesand eliminating the need to find a specific button in a sea of remotecontrol buttons. In addition to voice recognition, some systemsincorporate gesture control, whereby a small camera on a television, forexample, captures viewer gestures to identify an action requested by theviewer, such as to turn the television volume down.

To overcome problems with presence determination as an element of aneffective media measurement system, disclosed herein are presence andauthentication systems and methods that, in an embodiment, use audio andvideo fingerprinting to detect and confirm the presence of a viewer at amedia device, particularly a shared media device such as a television.The use of such audio or video biometric fingerprinting, eitherseparately or in combination, presents a largely passive solution to theproblems noted above above.

Video fingerprinting, in particular, may operate to determine thepresence of individual viewers among a group of viewers. Videofingerprinting may provide depth of field information, which helps toisolate a viewer from the background. Facial recognition, which mayinclude a depth of field component, may be used as one aspect of videofingerprinting to determine the presence of a specific viewer. Inaddition, gesture analysis, which may provide for some kind of“pass=gesture” as well as basic physical attributes such as size andpossibly gait analysis, may be used as part of the video fingerprintingprocess—for example, by pointing a camera at the room's doorway andmeasuring the height of viewers as they walk into or out of the room.

Some television platforms and some set top boxes (STBs) incorporateprocessors that are capable of passive viewer identity processes forpresence and authentication purposes by capturing certain biometricinformation about the viewer, such a video or audio fingerprint of theviewer, as noted above. However, in addition to the processingcapability, and any required identity programming, these media devicesrequire some additional hardware, firmware, and/or software to capturethe biometric information. Some televisions are being produced withsmall cameras capable of capturing such information. The camerasgenerally have a wide lens and are directed toward the intended viewingaudience. Similarly, some television incorporate microphones that may beused for audio fingerprinting of viewers.

With respect to viewer identity, using a video camera connected to themedia device, facial recognition software may be employed as a tool indetecting the number and identities of viewers in real time. A mediadevice with a video camera (or still camera) may capture the facialimages of viewers in a viewing location (e.g., in a room such as theviewers' living room) as the viewers come and go, and may use thisinformation to personalize or improve the viewers' viewing experienceand to better measure the viewers' viewing history.

Generally, facial recognition is a computer-based system forautomatically identifying or verifying a person from a digital image ora video frame. Recognition algorithms include at least two mainapproaches. A geometric approach looks at distinguishing facialfeatures, while a photometric approach is a statistical approach thatdistills an image into values and compares the values with templates toeliminate variances in order to find a match. The selected facialfeatures may be, for example, the relative position, size, and/or shapeof the eyes, nose, cheekbones, and jaw. These salient features then arecompared with features of other images in a data set to locate matchingfeatures.

When using a computerized multimedia device such as a smart television,a viewer may choose to initially associate his face with his identity.For example, the viewer may sign on to a service such as a streamingvideo service and register an image of his face with the service. Whenthe viewer subsequently access his account, the multimedia device maycapture a current image of the viewer's face and use that image toverify the presence and authenticate the identity of the viewer. Oneadvantage of this approach is that it is passive—that is, this approachdoes not require any subsequent log-in/log-out actions by the viewer.This passive approach to determining viewer presence may enhance theviewer's viewing experience. Alternatively, the viewer may associate animage of his face with his identity when initially setting up the mediadevice—that is, the facial image is stored in a database on themultimedia device.

Even if a viewer chooses not to identify himself as above, knowing a“logical identity” (i.e., the facial recognition software recognizes thesame viewer as these other times) or a general category (e.g., a male asdistinguished from a female when a viewing location is known normally tobe populated only by males, may help identify the viewer, as describedbelow.

Accurate measurement of media consumption metrics may hinge on more thanjust viewing history. In particular, media consumption measurement maybe improved by accurate determination of viewer presence when multipleviewers are present, and when multiple types of media devices are inuse. For example, although one viewer in a household may watch aparticular sporting event on a home media device (e.g. a television),commercials related to the sporting event may be of limited interest toother viewers in the household. This is where the use of identity alongwith viewing history becomes valuable. Viewing history may be recordedwith respect to the identities of the viewers present in the room duringdifferent shows. Each viewer may have a different history.

In summary, gathering a viewing history for each viewer based on apassive, auto-recognized identity, inferring demographic or viewer'sinterests information based on past shows that each individual haswatched, possibly combined with other program provider accountinformation (e.g., search history or profile and other informationavailable at a social networking website) provides a great amount ofhighly relevant information in selecting better ad choices. Some mediadevices can dynamically display the ads that are relevant to the viewersthat are present based on this information.

In embodiments disclosed herein, viewers may explicitly identifythemselves each time they sit in front of the television, rather thanautomatically being detected by a video camera. This type ofidentification is essentially “logging in” to watch television. Explicitincentives for this process may be provided such as special deals onadvertised products for watching a particular show, or restrictingaccess to certain shows.

In other embodiments disclosed herein, viewer presence may begin bypicking up the viewers' faces with a camera (e.g., a wide-angledfront-facing camera) embedded in or mounted on the television or somecomponent of the television, and using facial recognition, matching theviewers' faces with faces associated in some way to online socialnetworking profiles.

FIGS. 1A-1C illustrate example environments in which viewer presence maybe determined and viewer identity verified (authentication) usingpassive presence and passive and active authentication mechanisms.

FIG. 1A illustrates an example environment in which personal analyticsand usage controls may be implemented. In FIG. 1A, environment 10includes viewing locations 20, sponsor 40, and program provider 60, allof which communicate using communications network 50. Although FIG. 1Ashows these entities as separate and apart, at least some of theentities may be combined or related. For example, the sponsor 40 andprogram provider 60 may be part of a single entity. Other combinationsof entities are possible.

The viewing location 20 includes first media device 24 and second mediadevice 26 through which viewers 22 are exposed to media from sponsor 40and program provider 60. A viewing location 20 may be the residence ofthe viewer 22, who operates media devices 24 and 26 to access, throughrouter 25, resources such as Web sites and to receive televisionprograms, radio programs, and other media. The media devices 24 and 26may be fixed or mobile. For example, media device 24 may be an Internetconnected “smart” television (ITV); a “basic” or “smart” televisionconnected to a set top box (STB) or other Internet-enabled device; forexample. In an embodiment, the media device 24 includes biometricinformation capture devices and systems, which are described in detailwith respect to fire 1B and FIGS. 2A and 2B. Media device 26 may be atablet, a smart phone, a laptop computer, or a desk top computer, forexample. The media devices 24 and 26 may include browsers. A browser maybe a software application for retrieving, presenting, and traversingresources such as at the Web sites. The browser may record certain datarelated to the Web site visits. The media devices 24 and 26 also mayinclude applications. A viewer 22 may cause the media devices 24 or 26to execute an application, such as a mobile banking application, toaccess online banking services. The applications may involve use of abrowser or other means, including cellular means, to connect to theonline banking services.

The viewing location 20 may include a monitor 27 that records andreports data collected during exposure of sponsored content segments 42and programs 62 to the viewer 22. The example monitor 27 may beincorporated into router 25 through which certain media (e.g.,Internet-based content) received at the viewing location 20 passes.

The sponsor 40 operates server 44 to provide sponsored content segmentsthat are served with programs 62 provided by the program provider 60.For example, the server 44 may provide sponsored content segments toserve with broadcast television programming. The sponsored contentsegments 42 may include audio, video, and animation features. Thesponsored content segments 42 may be in a rich media format. The sponsor40 may provide a promotional campaign that includes sponsored contentsegments to be served across different media types or a single mediatype. The cross-media sponsored content segments 42 may becomplementary; that is, related to the same product or service.

The network 50 may be any communications network that allows thetransmission of signals, media, messages, voice, and data among theentities shown in FIG. 1, including radio, linear broadcast(over-the-air, cable, and satellite) television, on-demand channels,over-the-top media, including streaming video, movies, video clips, andgames, and text, email, and still images, and transmission of signals,media, messages, voice, and data from a media device to another mediadevice, computer, or server. The network 50 includes the Internet,cellular systems, and other current and future mechanisms fortransmission of these and other media. The network 50 may be both wiredand wireless. The network 50 may be all or a portion of an enterprise orsecured network. In an example, the network 50 may be a virtual privatenetwork (VPN) between the program provider 60 and the media devices 24and 26. While illustrated as a single or continuous network, the network50 may be divided logically into various sub-nets or virtual networks,so long as at least a portion of the network 50 may facilitatecommunications among the entities of FIG. 1A.

The program provider 60 delivers programs for consumption by the viewer22. The programs 62 may be broadcast television programs. Alternately,the programs 62 may be radio programs, Internet Web sites, or any othermedia. The programs 62 include provisions for serving and displayingsponsored content segments 42. The program provider 60 may receive thesponsored content segments 42 from the sponsor and incorporate thesponsored content segments into the programs 62. Alternately, theviewer's media devices may request a sponsored content segment 42 whenthose media devices display a program 62.

The program provider 60 operates server 66 to serve programs and toimplement usage control system 200. The system 200 may collectinformation related to programs 62 displayed at the media devices 24 and26. The system 200 may provide an interface that allows the viewer 22 toestablish usage controls.

FIG. 1B illustrates aspects of the environment 10 of FIG. 1A,emphasizing viewer presence and authentication features. In FIG. 1B,media device 24 (an Internet-connect smart television) at viewinglocation 20 is shown to include camera 305, microphone 306, and audiorecognition system 312 and video recognition system 314. Some of thesecomponents of the television 24 may form at last a part of a presenceand authentication system 300 (see FIG. 3) whereby viewers at theviewing location 20 may have their presence detected and theiridentities authenticated. The television 24 receives programming andadvertisements through gateway 25, and provides measurement data toanalytics service 70 through the gateway 25. In an embodiment, thegateway is a router (e.g., router 25 of FIG. 1A). In an aspect, therouter 25 may be configured to log certain information related toprograms viewed on and advertisements served at the television 24. Therouter 25 may pass this information to the analytics service 70. Therouter 25 also may pass viewer presence and authentication informationto the analytics service 70.

Three viewers 22 are shown at the viewing location 20, each viewingprogramming on the television 24. The feature extraction and analysissystem may obtain certain biometric information about the viewers 22,such as video and audio fingerprint information and use the informationto determine, within some confidence level, how many viewers 22 are infront of the television 24, what the identities of the viewers 22 are,which viewer is interacting with the television 24 (e.g., changingchannels with a remote control) and other information. Note that anyimage/audio processing occurs locally, only. Certain aspects of theprocessed information, but not any viewer images or audio, and noinformation that may be used to identify a viewer, then may be suppliedto the analytics service 70.

FIG. 1C illustrates additional aspects of the environment 10 of FIG. 1A.In FIG. 1C, viewing location 20′ is shown as a residence of four viewers22A-22C2. The viewing location 20′ includes three separated viewinglocations (or rooms) 20A-20C. Each such viewing location has installedtherein a fixed media device, or Internet-enabled smart television 24 i.In addition to the televisions 24 i, the viewing location 20′ alsoincludes, in location 20B, smartphone 26B and in viewing location 20C,tablet 26C. All these media devices may be connected, by wired orwireless mechanisms (e.g., signal path 23), to router 25, which in turnis connected to analytics server 82 over network 50.

The televisions 24 i each may include components to determine presenceand authenticity of the viewers (i.e., the same television components asshown in FIG. 1B). As is clear from FIG. 1C, a viewer in location 20B(i.e., viewer 22B) cannot be viewing the television 24C. Therefore, ifthe presence and authentication components of television 24C were toindicate the presence of the viewer 22B in the location 20C, thatpresence indication would be erroneous.

In operation, the televisions 24 i of FIG. 1C may detect when a viewer22 enters or leaves a room, may determine a number of viewers in a room,and may determine which of multiple viewers 22 in a room is operating atask such as issuing orders to the television 24 i. Note that theviewers need not be signed-in to the televisions 24 i, or to any othermedia device. The presence and authentication system may operate in acompletely passive mode. Alternately, the system may include activefeatures, including active authentication features such as sign-in andpassword entry. An example of a presence and authentication system, asinstantiated locally at the viewing location 20′, is described in moredetail with respect to FIG. 3.

In executing the processes of FIGS. 1A-1C, and as otherwise disclosedherein, individual viewer and household demographic data, Internetactivity, and television viewing data, for example, may be collected andused. In situations in which the systems disclosed herein may collectand/or use personal information about viewers, or may make use ofpersonal information, the viewers may be provided with an opportunity tocontrol whether programs or features collect viewer information (e.g.,information about a viewer's social network, social actions oractivities, profession, a viewer's preferences, or a viewer's currentlocation), or to control whether and/or how to receive media, includingadvertisements, from an server that may be more relevant or of interestto the viewer. Furthermore, where the control process involves detectionof personal features, such as facial features capture through facialrecognition) the viewers consent to capture and analysis of thefeatures. In addition, certain data may be treated in one or more waysbefore it is stored or used, so that personally identifiable informationis removed. For example, a viewer's identity may be treated so that nopersonally identifiable information can be determined for the viewer, ora viewer's geographic location may be generalized where locationinformation is obtained (such as to a city, ZIP code, or state level),so that a particular location of a viewer cannot be determined. Thus,the viewer may have control over how information is collected about theviewer and used by a server.

FIG. 2A illustrates an example measurement system instantiated onanalytics server 82. The system includes processor 84, memory 86,input/output 88, and database 85. The database 85 may be anon-transitory computer-readable storage medium having encoded thereonmeasurement system 200. The processor 84 loads the machine instructionsinto memory 86 and executes the machine instructions to provide personalanalytics and usage controls functions. The I/O 88 allows the server 70to communicate with other entities such as the server 44.

The system 200 may, in an embodiment, perform feature extraction andanalysis processes to determine the presence and authenticity of viewersat a viewing location such as the location 20′ of FIG. 1C.

FIG. 2B illustrates example components of a media measurement system200, implemented on the analytics server 82 of FIG. 2A. In FIG. 2B,system 200 includes viewer presence engine 210, viewer authenticationengine 220, validation engine 230, measurement engine 240, and reportingengine 250.

The viewer presence engine 210 and the viewer authentication engine 220may receive summary information from.

The validation engine 230 may, based on the presence and authenticationinformation generated locally, determine that the confidence level ofthe information is sufficient to use in measurement analysis. Forexample, if the confidence level is 90 percent or higher, the validationengine 230 may allow use of the data in performing media measurements.

The measurement engine 240 determines various media consumption metricssuch as reach, incremental reach, TRP, and other media consumptionmetrics. The repotting engine 250 may repots the media consumptionmetrics to interested parties such as the sponsor 40 and programprovider of FIG. 1C.

FIG. 3 illustrates an example of television components 24′, includingpresence and authentication system 300 that may be installed orimplemented on a television 24. The components 24′ include database 301,memory 302, processor 303, and I/O 304.

The database 301 includes a non-transitory computer-readable storagemedium on which is encoded system 300. The system 300 may be loaded intomemory 302 and executed by processor 303. I/O may be used forman-machine communications between a viewer 22 and the processor 303.The processor 303 also receives inputs (raw or processed data) from thecamera 305 and the microphone 306.

The presence and authentication system 300 includes featureextraction/recognition engine 310, location engine 320, use matrixengine 330, presence engine 340, and authentication engine 350.

The engine 310 includes audio module 312 and video module 314. The audiomodule 312 receives raw or processed audio data captured by themicrophone 306 and produces a voice fingerprint, which is provided tothe presence engine 340. The video module 314 receives raw or processedvideo data captured by the camera 305 and produces a video fingerprint,which is provided to the presence engine 340.

The location engine 320 may receive location information related to oneor more of the viewers 22 i (see FIG. 1C). For example, viewer 22C1 maybe determined to be operating tablet 26C, either because the viewer 22C1has logged on to a tablet service, or because a camera in the television24C has detected a viewer operating a tablet. Other mechanisms forlocating the tablet 26 c may be used. The tablet 26C may be geo-locatedby, for example, a GPS system, which may locate the tablet 26 c to thespecific viewing location (room) 20C, or at least to the larger, overallviewing location 20′. Any location information for a viewer may bepassed to the presence engine 340.

The use matrix engine 330 constructs a three-dimensional use matrixconsidering an initial condition of media devices and viewers at theviewing location 20′. That is, the matrix would show the location ofeach media device in each room, and the location of each viewer in eachroom. The engine 330 may update the matrix as the number and identity ofmedia devices in the viewing location 20′ changes, and as viewers comeand go (both on a semi-permanent basis). The engine 330 then maypopulate the latest iteration of the matrix to reflect real timepositioning of media devices and viewers as best determined by theengines 310 and 320. For example, the engine 330 may populate the matrixwith values corresponding to the arrangement of media devices andviewers shown in FIG. 1C. the engine 330 then makes the populated matrixavailable to the presence engine 340.

The presence engine 340 determines a probability that a particularviewer and a particular media device are in a specific room based on itsreceived inputs. For example, the probability that television 24C is inroom 20C is 100 percent, but the probability that the viewer 22C1 is inroom 20C may be ⅓ or greater (assuming the viewer 22C1 is in the viewinglocation 20′ at all). The probability that the viewer 22C1 is in room20C may be increased based on audio and video fingerprint informationreceived from the engine 310 and location information received from theengine 320.

The presence engine 340 may, when multiple viewers are present, not beable to distinguish between the multiple viewers. In that case, theengine 340 may report the presence of multiple viewers. However, if, forexample, viewer 22B was known with a 90 percent confidence level, to bein room 20B, the engine 340 could use this information to betterindicate the composition of the viewers in room 20C (i.e., possiblyviewers 22A, 22C1, 22C2; not viewer 22B).

The presence engine 340 may provide the presence information to theanalytics server 82 and to the authentication engine 350.

The authentication engine 350 may provide for passive and activeauthentication processes. For example, the engine 350, knowing thatviewer 22A is short, might receive from the presence engine 340 a videofingerprint conforming to the shape and size of a short person. Inaddition, the engine 350 may receive a voice fingerprint conforming tothe specific voice patterns of the viewer 22A. By this and othercross-validation processes, the engine 350 may passively authenticatethe identity of viewers 22 at the viewing location 20′.

The authentication engine 350 also may provide active authenticationfunctions, such as requesting and/or receiving a viewer sign on byviewer identification and password entry, for example.

FIG. 4 is a flow chart illustrating an example viewer presence andauthentication process as executed by the systems of FIGS. 2A and 2B. At410, the process 400 receives TV on Signal. At 420, the process 400detects viewer. At 500, the process 400 determines viewer(s) present. At600, the process 400 authenticates viewers present. At 700, the process400 determines viewer actions. At 800, the process 400 reports toanalytics service.

FIG. 5A is a flow chart illustrating an example viewer presence andauthentication process as executed by the systems FIGS. 2A and 2B. At505, the process 500 determines viewer motion. At 510, the process 500performs general categorization process and match to known viewerprofile. At 515, the process 500 records audio/video fingerprint data.At 520, the process constructs fingerprints. At 525, the process 500determines whether there is a fingerprint match. If there is nofingerprint match at 525, the process 500 returns to 510/515. If thereis a fingerprint match at 525, the process 500 proceeds to block 530.

FIG. 5B is a flow chart illustrating an example viewer presence andauthentication process as executed by the systems FIGS. 2A and 2B. At530, the process 500 receives viewer location data. At 535, the process500 determines whether there is a location data match TV matrix. If theprocess 500 determines there is no location data match TV matrix at 535,the process 500 returns to block 530. If the process 500 determines at535 there is a location data match TV matrix, the process proceeds to540. At 540, the process 500 computes presence probability withconfidence level. At 545, the process 500 determines whether theprobability and C/I is greater than a threshold. If the probability andC/I is not greater than the threshold at 545, the process 500 returns toblock 540. If the probability and C/I is greater than the threshold at545, the process 500 proceeds to block 550.

FIG. 5C is a flow chart illustrating an example viewer presence andauthentication process as executed by the systems FIGS. 2A and 2B. At550, the process 500 passes presence information to authenticationengine and analytics.

FIG. 6 is a flow chart illustrating an example viewer presence andauthentication process as executed by the systems FIGS. 2A and 2B. At605, the process 600 performs passive authentication. At decision block610, the process 600 determines whether the viewer P is greater than orequal to X %. If the viewer authentic P is greater than or equal to X %at 610, then the process 600 proceeds to block 700. If the viewer P isnot greater than or equal to X % at 610, then the process 600 proceedsto 615. At 615, the process 600 performs active authentication. At 620,the process 600 determines whether viewer is authentic. If the process600 determines the viewer is not authentic at 620, then the process 600returns to block 605/615. If the process 600 determines the viewer isauthentic at 620, then the process 600 proceeds.

FIG. 7 is a flow chart illustrating an example viewer presence andauthentication process as executed by the systems FIGS. 2A and 2B.Process 700 includes detecting viewer voice gesture commands, detectingcross-media device operation, and logging commands/operations. Process700 proceeds to block 800 as depicted in FIG. 4.

FIG. 8 is a flow chart illustrating an example viewer presence andauthentication process as executed by the systems FIGS. 2A and 2B. At805, process 800 receives presence/authentication data for viewer. At810, the process 800 determines whether a threshold has been exceeded.If the process 800 determines the threshold has not been exceeded at810, the process 800 returns to block 805. If the process 800 determinesthe threshold has been exceeded at 810, the process 800 proceeds to 820.At 820, the process 800 performs metric measurements. At 825, theprocess 800 reports metrics.

Certain of the devices shown in the herein described figures include acomputing system. The computing system includes a processor (CPU) and asystem bus that couples various system components including a systemmemory such as read only memory (ROM) and random access memory (RAM), tothe processor. Other system memory may be available for use as well. Thecomputing system may include more than one processor or a group orcluster of computing system networked together to provide greaterprocessing capability. The system bus may be any of several types of busstructures including a memory bus or memory controller, a peripheralbus, and a local bus using any of a variety of bus architectures. Abasic input/output (BIOS) stored in the ROM or the like, may providebasic routines that help to transfer information between elements withinthe computing system, such as during start-up. The computing systemfurther includes data stores, which maintain a database according toknown database management systems. The data stores may be embodied inmany forms, such as a hard disk drive, a magnetic disk drive, an opticaldisk drive, tape drive, or another type of computer readable media whichcan store data that are accessible by the processor, such as magneticcassettes, flash memory cards, digital versatile disks, cartridges,random access memories (RAM) and, read only memory (ROM). The datastores may be connected to the system bus by a drive interface. The datastores provide nonvolatile storage of computer readable instructions,data structures, program modules and other data for the computingsystem.

To enable human (and in some instances, machine) user interaction, thecomputing system may include an input device, such as a microphone forspeech and audio, a touch sensitive screen for gesture or graphicalinput, keyboard, mouse, motion input, and so forth. An output device caninclude one or more of a number of output mechanisms. In some instances,multimodal systems enable a user to provide multiple types of input tocommunicate with the computing system. A communications interfacegenerally enables the computing device system to communicate with one ormore other computing devices using various communication and networkprotocols.

The preceding disclosure refers to flowcharts and accompanyingdescriptions to illustrate the embodiments represented in FIGS. 7 and 8.The disclosed devices, components, and systems contemplate using orimplementing any suitable technique for performing the stepsillustrated. Thus, FIGS. 7 and 8 are for illustration purposes only andthe described or similar steps may be performed at any appropriate time,including concurrently, individually, or in combination. In addition,many of the steps in the flow charts may take place simultaneouslyand/or in different orders than as shown and described. Moreover, thedisclosed systems may use processes and methods with additional, fewer,and/or different steps.

Embodiments disclosed herein can be implemented in digital electroniccircuitry, or in computer software, firmware, or hardware, including theherein disclosed structures and their equivalents. Some embodiments canbe implemented as one or more computer programs, i.e., one or moremodules of computer program instructions, encoded on computer storagemedium for execution by one or more processors. A computer storagemedium can be, or can be included in, a computer-readable storagedevice, a computer-readable storage substrate, or a random or serialaccess memory. The computer storage medium can also be, or can beincluded in, one or more separate physical components or media such asmultiple CDs, disks, or other storage devices. The computer readablestorage medium does not include a transitory signal.

The herein disclosed methods can be implemented as operations performedby a processor on data stored on one or more computer-readable storagedevices or received from other sources.

A computer program (also known as a program, module, engine, software,software application, script, or code) can be written in any form ofprogramming language, including compiled or interpreted languages,declarative or procedural languages, and it can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, object, or other unit suitable for use in a computingenvironment. A computer program may, but need not, correspond to a filein a file system. A program can be stored in a portion of a file thatholds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub-programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

We claim:
 1. A system, comprising: one or more servers comprising one ormore processors and memory; an interface of the one or more servers toreceive an audio-based input detected by a microphone of a remotecomputing device; a feature recognition engine of the one or moreservers to generate an audio fingerprint based on the audio-based input;the interface to receive an input from a second computing devicedifferent from and associated with the remote computing device; anauthentication engine of the one or more servers to: determine, based onthe audio fingerprint, a probability level that the audio-based input isfrom a registered user of the remote computing device having a profilestored on the one or more servers; increase, responsive to the inputreceived from the second computing device different from and associatedwith the remote computing device, the probability level determined basedon the audio fingerprint; identify, responsive to the probability levelincreased by the input received from the second computing device andbeing above a predetermined threshold, an action identified within theaudio-based input, the action corresponding to a request by theregistered user to activate or control a feature of the remote computingdevice; select a content item based on the action identified within theaudio-based input and the profile of the registered user; and transmit,to the remote computing device via the interface, the content item tocause the remote computing device to present the content item via anoutput interface of the remote computing device.
 2. The system of claim1, comprising: the interface to receive an image comprising a facialfeature; the feature recognition engine to generate a facial recognitionscore based on a comparison between the image and a preregistered image;and the authentication engine to determine the probability level basedon the facial recognition score.
 3. The system of claim 1, comprising:the authentication engine to determine the probability level is belowthe predetermined threshold; the interface to transmit a request for anactive authentication; and the authentication engine to increase theprobability level above the predetermined threshold responsive toreceiving the active authentication.
 4. The system of claim 3, whereinthe request for the active authentication comprises a request for apassword.
 5. The system of claim 1, comprising: the authenticationengine to: determine a category associated with the audio fingerprint;and determine the probability level based on the category associatedwith the audio fingerprint.
 6. The system of claim 1, comprising: theinterface to receive an identification of a location associated with theremote computing device; and the authentication engine to determine theprobability level based on the identification of the location associatedwith the remote computing device.
 7. The system of claim 1, comprising:the interface to receive the input received from the second computingdevice associated with the remote computing device, the input indicatinga location; the authentication engine to: determine a probability theremote computing device is located at the location; and determine theprobability level based on the probability the remote computing deviceis located at the location.
 8. The system of claim 1, comprising: theinterface to receive biometric data from the second computing device;the authentication engine to determine the probability level based onthe biometric data.
 9. The system of claim 1, comprising: the interfaceto receive a video-based input detected by the remote computing device;the feature recognition engine to generate a video fingerprint based onthe video-based input.
 10. The system of claim 9, wherein thevideo-based input comprises input from a plurality of users.
 11. Amethod, comprising: receiving, by one or more servers, an audio-basedinput detected by a microphone of a computing device remote from the oneor more servers; generating, by a feature recognition engine of the oneor more servers, an audio fingerprint based on the audio-based input;determining, by an authentication engine of the one or more servers,based on the audio fingerprint, a probability level that the audio-basedinput is from a registered user of the computing device having a profilestored on the one or more servers; receiving, by the authenticationengine, input from a second computing device different from andassociated with the computing device; increasing, by the authenticationengine responsive to the input received from the second computing devicedifferent from and associated with the remote computing device, theprobability level determined based on the audio fingerprint;identifying, by the authentication engine and responsive to theprobability level increased by the input received from the secondcomputing device and being above a predetermined threshold, an actionidentified within the audio-based input, the action corresponding to arequest by the registered user to activate or control a feature of thecomputing device; selecting, by the authentication engine, a contentitem based on the action identified within the audio-based input and theprofile of the registered user; and transmitting, from the one or moreservers to the computing device, the content item to cause the computingdevice to present the content item via an output interface of thecomputing device.
 12. The method of claim 11, comprising: receiving, bythe one or more servers, an image comprising a facial feature;generating, by the feature recognition engine, a facial recognitionscore based on a comparison between the image and a preregistered image;and determining, by the authentication engine, the probability levelbased on the facial recognition score.
 13. The method of claim 11,comprising: determining, by the authentication engine, the probabilitylevel is below the predetermined threshold; transmitting, to thecomputing device, a request for active authentication; and increasing,by the authentication engine, the probability level above thepredetermined threshold responsive to receiving the activeauthentication from the computing device.
 14. The method of claim 13,wherein the request for the active authentication comprises a requestfor a password.
 15. The method of claim 11, comprising: determining, bythe authentication engine, a category associated with the audiofingerprint; and determining, by the authentication engine, theprobability level based on the category associated with the audiofingerprint.
 16. The method of claim 11, comprising: receiving, by theone or more servers, an identification of a location associated with thecomputing device; and determining, by the authentication engine, theprobability level based on the identification of the location associatedwith the computing device.
 17. The method of claim 11, comprising:receiving, by the one or more servers, the input received from thesecond computing device associated with the computing device, the inputindicating a location; determining, by the authentication engine, aprobability the computing device is located at the location; anddetermining, by the authentication engine, the probability level basedon the probability the computing device is located at the location. 18.The method of claim 11, comprising: receiving, by the one or moreservers, biometric data from the second computing device; determining,by the authentication engine, the probability level based on thebiometric data.
 19. The method of claim 11, comprising: receiving, bythe one or more servers, a video-based input detected by the computingdevice; generating, by the feature recognition engine, a videofingerprint based on the video-based input.
 20. The method of claim 19,wherein the video-based input comprises input from a plurality of users.